W9 2-22 328 


STRUCTURED ANALYSIS AND MODELING OF COMPLEX SYSTEMS 


David R. Strome, Ph.D. 

Systems Research Laboratories, Inc. 
P.O. Box 35482 

Brooks Air Force Base, Texas 78235 


Mathieu A. Dalrymple 

Systems Research Laboratories, Inc. 

P.O. Box 35482 

Brooks Air Force Base, Texas 78235 


ABSTRACT 


The Aircrew Evaluation Sustained Operations 
Performance (AESOP) facility at Brooks AFB, Texas, 
combines the realism of an operational environment with 
the control of a research laboratory. In recent studies 
we collected extensive data from Airborne Warning and 
Control Systems (AWACS) Weapons Directors subjected 
to low and high workload Defensive Counter Air 
scenarios. A critical and complex task in this 
environment involves committing a friendly fighter against 
a hostile fighter. Structured Analysis and Design 
techniques and computer modeling systems have been 
applied to this task as tools for analyzing subject 
performance and workload. This technology is being 
transferred to the Man-Systems Division of NASA/JSC 
for application to complex mission-related tasks, such as 
manipulating the shuttle grappier arm. 


INTRODUCTION 


Structured analysis and modeling are not new tools. 
They have been used for many years as aids in defining 
and analyzing systems, projects, products, and 
concepts. This paper discusses one application of these 
tools to a highly complex Air Force operational task, and 
the transfer of this technology to the Man-Systems 
Division of NASA. 

Modeling focuses our attention on the processes and 
relationships of a system. It allows us both to describe 
a system as it is and to predict system behavior when 
conditions or constraints are altered. The accuracy of 
the prediction depends on the complexity of the question 
and the detail and validity of the model. Although often 
thought of as a research method, computer modeling is 
used in many forms: financial systems are modeled with 
spreadsheets; projects are modeled with program 
management software; motion is modeled with 
specialized graphics systems. Research models use 
specialized programs to explore neural 
function, communications, strategies and tactics, human 
performance, and task loading. These are only a few of 
the multitude of applications. 


Creating a model is a simple concept but a demanding 
process. Using input from experts, the system being 
modeled must be broken down into detailed steps or 
processes, the relationships between the steps must be 
clearly defined, and the parameters that affect each 
process or relationship must be determined. The model 
is strengthened, verified, and validated by testing the real 
world system under a variety of conditions and 
comparing the data with the model’s results. This 
iterative procedure continues until the model reaches a 
level of description and prediction that meets the 
demands of the research. 

Structured analysis is a formalized process for 
developing the detailed information required to build the 
model. As a process, it is accomplished when any 
model is built, whether it is done formally or not. For 
simple systems models can be adequately defined with 
less rigorous methods. Complex tasks, on the other 
hand, benefit from a more structured technique that aids 
the analysis. Structured Analysis and Design Technique 
(SADT) [Marca and McGowan, 1988] is a widely 
recognized implementation of this concept. Originally a 
paper and pencil exercise, it is now appearing in 
software tools that provide basic consistency checks, 
easy modification, and presentation output. The basic 
building block of SADT is a box containing a descriptor 
of an event or process. This event has inputs, outputs, 
controls, and resources or mechanisms (Figure 1). 
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These blocks are linked together hierarchically in 
increasing detail until the model is sufficiently described. 
Structured analysis: 

♦ improves system definition. 

♦ improves problem understanding. 

♦ improves user/developer communication. 

♦ impacts system design. 

♦ smooths transition from analysis to design. 

Technical developments in computer hardware and 
software have supported attempts to marry these two 
tools into a single integrated package of analysis and 
modeling. As separate tools, the information entered 
into the structured analysis program has to be re- 
entered into the modeling software. To overcome this 
duplication of effort, software engineers are developing 
integrated systems that automatically port the structured 
information into selected modeling systems. 


AIR FORCE APPLICATION 


The Aircrew Evaluation Sustained Operations 
Performance (AESOP) facility is located in the Sustained 
Operations Branch of the Crew Technology Division at 
Brooks Air Force Base, San Antonio, Texas. It is a 
research facility dedicated to the following goals: 

♦ Develop and apply individual and team 

performance measures to evaluate crew 

interactions under sustained operations. 

♦ Develop and integrate tools to support research 

and development. 

♦ Transfer technology to operational 

environments. 

During 1989 the Air Force conducted a study in AESOP 
using 12 teams of Airborne Warning and Control 
Systems (AWACS) Weapons Directors (WDs). WDs 
provide control for Air Forces in their area of 
responsibility, committing friendly aircraft to missions 
such as: identification of unknown aircraft; engagement 
of hostile aircraft; search and rescue operations; mid-air 
refueling; and escort. Their operational environment is 
a complex combination of tactics and strategy, decision- 
making, and communication. 

Each team of three WDs was subjected to three high 
workload and three low workload defensive counter air 
scenarios, each lasting three and one-half hours. 
Among the collected operational data were: switch 
actions, key presses, communication channel usage, 
audio traffic, microphone activations, target locations, 
and mission critical events. Video tapes captured non- 
verbal, non-console subject interaction. In addition to 


operational measures, a battery of individual 
performance tests, surveys, and questionnaires was 
given throughout the course of the study. The resulting 
data is being continually analyzed to develop and test 
individual and team performance measures directly 
related to operational tasks. 

Part of the ongoing development of tools to assist in this 
research involved selecting a complex WD task for 
structured analysis and modeling. The purpose was to 
evaluate several software systems while determining the 
applicability of the tools to our environment. The data 
from real subjects provided a testbed for the validity of 
the model. 

Committing a friendly fighter to a mission against a 
potentially hostile, or known hostile, target is a primary 
WD task. The first two levels of the SADT analysis, 
created using Idefine (Wizdom Systems, Inc.), are shown 
below (Figures 2,3) 


Figure 2 

At the top level (Figure 2) we define the broadest 
description of the task. Targets on the computerized 
radar screen are the INPUTS to the task. The WDs and 
their team strategy are MECHANISMS for completing the 
task. The Air Tasking Order (ATO) f tactics, and 
assigned resources are the CONTROLS 
(CONSTRAINTS) on conducting the task. When the task 
is complete the possible OUTPUTS include a target 
being out of Area Of Responsibility, a scrambie/airborne 
order requesting additional resources, a commit of 
existing resources to an intercept mission , or committing 
to some other mission such as search and rescue or 
escort. The next part of the hierarchy (Figure 3) divides 
this broad analysis into the next level of detail. 

This level has the same overall inputs, controls, 
mechanisms, and outputs; but the task is now 
subdivided into identifying targets, sorting threats , sorting 
friendlies , and committing to a mission. These tasks are 
connected functionally with outputs becoming inputs or 
controls to other steps as necessary. Each of these 
boxes is then subdivided in the same manner at the next 
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Figure 3 


level of the hierarchy, and so on until reaching the 
required depth of detail. 

The finest level of detail is entered into the modeling 
software. Time to completion, probability of a particular 
path, and other selected parameters are defined for 
each task to complete the definition of the model. 
Figure 4 is a typical output, graphing the frequency 
distribution of time required to complete the commit task. 


Commit Activity Execution Time Frequency Distribution 



Figure 4 


The average completion time is around 37 seconds, 
which matches well the average time collected in actual 
scenarios. Changes in the model result in predictable 
shifts in this average time. However several aspects of 
the performance of real WDs are not accounted for by 
the current model. The model tends to have much less 
variation than human operators. It also does not 
account well for very slow or very fast times, which may 
be due to inattention, fatigue, anticipation, distraction, or 
other external causes. Finally it does not partition 
realistically between component phases of the overall 
task, such as a decision phase where the WD is 


deciding what to do, and the action phase where 
switches and keys are pressed and information is 
communicated. Our experience suggests that the 
accuracy required to adequately describe and predict 
real world behavior will demand that these aspects be 
accounted for. To do so will require models that provide 
true parallel processing and accept algorithms for 
dynamics such as fatigue, anticipation, and distraction. 


TECHNOLOGY TRANSFER 


Working under the guidelines of a memorandum of 
understanding, the Sustained Operations Branch is 
working with the Crew Interface Analysis Section of the 
Man Systems Division at NASA/Johnson Space Center 
to integrate these tools into the space operations and 
research environment. The first task to be analyzed and 
modeled by NASA is the operation of the Remote 
Manipulator System at both normal and reduced gravity. 


CONCLUSION 


The application of these tools to both Air Force and 
NASA operational tasks has emphasized their usefulness 
in defining and understanding complex systems. Future 
use of structured analysis and modeling in the AESOP 
will aid in; 

4 Determining critical performance elements in 
complex tasks. 

♦ Predicting performance effects of fatigue, 

decision aids, and drugs. 

♦ Defining and testing new training strategies. 

4 Developing meaningful individual and team 
performance measures. 
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